ZXXrUMENT RESUME 



ED 333 574 



luA 023 086 



AUTHOR 
TITLE 



cuttance, Peter 

Monitoring Educational Quality through Performance 
Indicators for School Practice* 
Apr 91 

23p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (Chicago , 
IL, April 3-7, 1991) . 

Speeches/Conference Papers (150) — Reports - 
Research/Technical (143) 



PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MFOl/PCOl Plus Postage. 

^Educational Assessment; ^Educational Quality; 
Elementary Secondary Education; ^Evaluation Criteria; 
Foreign countries; Program Evaluation; School 
Effectiveness 

*Australia (South Australia) ; strategic Planning 



ABSTRACT 



Issues in the monitoring of educational quality 



through performance indicators are discussed in this paper. The first 
section discusses the management of quality in educational systems 
and identifies the general functions of performance indicators as 
quality assurance # development, and accountability. Indicator systems 
are discussed in the following sections, and the relationships among 
indicators, development, and accountability are addressed in the 
context of the South Australian educational review system- The 
primary purpose of the review is to report on performance with 
respect to specific practices across the system as a whole. Effective 
practice reviews are conducted in four domains: teaching and 
learning; organization and management; ethos and culture; and social 
justice. The final sections discuss the use of performance indicators 
for development and quality management purposes as part of a broad 
framework for monitoring educational quality. An. educational review 
program based on group discussion, interviews, and observation 
assessed teaching and learning in 80 classrooms and management and 
organization in 17 schools. Findings indicate that the teaching and 
learning practices were stronger than those for management and 
organization. The recommendation is made for strengthening curriculum 
aspects of schooling, particularly planning, development, and review; 
delivery strategies; staff development; and management and review of 
development. Two figures are included. (8 references) (LMI) 
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Managing Quality In Education Systems 



latroduction 

This first section discusses the management of quality in education systems. 
Indicator systems are discussed in the section that follows and the relationship 
between indicators, development and accountability is addressed in the context of a 
system of education review that has recendy been established in South Australia. 
The final sections of the paper discuss the development of performance indicators 
to serve both development and quality management purposes. These indicators 
have been developed as part of a broad framework for continuously monitoring and 
reviewing the quality of the education provided for students. Information from the 
indicators of effective school practices and functioning is discussed in the final 

Quality Assurance or Quality Control? 

It is important to distinguish between quality management and quality control. 
Centralised systems have tended lo depend on quality control as their primary 
means of quality assurance. Quality control in such systems can be characterised 
as: 

• input orientated - as exemplified by its focus on administrative controls 
over inputs to the system. Most of the form-filling that we undertake in 
bureaucratic systems is designed to meet the purpose of control over 
inputs. 

• inspection orientated - that is, inspection of the outputs to select out items 
which do not meet pre-defined pnxJua standards. 

Quality Control Systems 

Most education systems have exercised some control over output. Typically 
this has taken the form of inspection of the outputs from the system. Inspection in 
education systems has taken one of two forms: the system-wide testing of student 
achievement, or the use of professional inspectors to assess the standard of student 
work (HMI, for example). Inspection systems are based on the principle of 
assessing whether the system 'got it right'. Inspection systems usually also attempt 
to provide some basic additional information so that operations may be changed to 
get it right next time. 
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Inspection systems are not primarily designed to assure quality outcomes. 
They attempt to 're-work* defective 'goods* through such processes as retaining 
students until they have reached a particular standard or by the non-certification of 
some students. In terms of quality control, uncertified students, that is those who 
leave school without any formal credentials, are a parallel in the education system to 
defective goods produced by a manufacturer and sold as 'seconds'. The 
manufacturer docs not stamp such goods with their (quality) label. The existence 
of such unlabellcd lines of products from manufacturers is usually a symptom of 
poor quality management 

Quality Manageinent Systems 

The alternative to input controls and inspection systems are quality 
management systems. Quality management systems attempt to assure quality 
through the introduction of appropriate processes for the management and 
monitoring of operations. That is, they attempt to integrate the process of work 
with the necessary mechanisms for assuring quality at each stage of the process. 
Plans indicating clear objectives and strategies of how those are going to achieved, 
review systems and audit procedures are basic components of quality management 
approaches. 

To be effective such approaches must be a part of the every day activity of 
work groups. The system itself also requires a quality assurance mechanism. 
Systemic quality assurance is based on audit and review of operational units to 
ensure that they have the appropriate quality management structures, and the direct 
monitoring of outcomes. At the school-level this requires the introduction and 
maintenance of systems to continuously monitor the effectiveness of strategies and 
practices — the function of action plans. 

Monitoring systems should be designed to provide frequent assessments of 
progress so that action to correct unintended effects can be taken at the first sign of 
things not working the way they were expected to. It is more important that the 
monitcning system be simple and that observations be made frequentiy, than always 
be guaranteed to give the right answer. 

Although the quality of outcomes themselves need to be continuously 
monitored, the system in referred to as quality management because it aims to 
assure quality outcomes by insuring that the processes implemented lead to the 
intended intermediate outcomes in every aspect of the system's operation. 

Quality Assurance in Devolved Systems 

Devolved systems require the introduction of quality assurance processes, 
because they do not have recour>e to the quality control mechanisms provided by 
centralised administrative systems. The lack of such control systems is not 
considered a disadvantage, however, because the emphasis they place on inputs (as 



is exemplified by the four signatures required, one for each step in the process, 
before a requisition reaches 'Stores Branch') is now considered inappropriate and 
counter productive. Management perspectives now almost universally emphasise 
the need to focus not on inputs, but on outputs and outcomes. This approach is 
widely accepted within tl»e Education Department of South Australia, although there 
is still some distance to travel before it is fully implemented thiDughout all units and 
functions. 

Effective management requires specific objectives, clear responsibility and 
quality assurance systems designed to access progress in meeting these objectives. 
Unclear or undefined objectives do not provide for clear delegation of 
responsibility. Individuals and groups can only be properly held accountable for 
achieving, or not achieving, specific objectives. In the absence of specific and clear 
objectives accountability becomes vague and unenforceable. Bureaucratic 
centralised systems fall back on the process of checking whether administrative 
instructions have been followed, rather than assessing how well educational 
objectives have been met. A centralised system necessarily operates by rules, set 
procedures and statute in order to reduce the number of problematic decisions that 
officials at the centre have to take. In the case of the education system these rules 
are of major as well as of minor scale and they militate against the overall co- 
ordination and responsiveness of the system. 

These rules are frequently not only inappropriate and inflexible: they also create 
a multitude of administrative forms and bureaucratic arrangements which take time 
to produce, time to update, time *o understand, and time to comply with (or avoid). 
Such an approach leads to a great deal of effort being devoted at all levels to rule 
making and rule breaking rather than to establishing goals and parameters for 
achievements and then setting out to achieve these. 

Audit and review processes are fundamental to the delivery of high quality 
outcomes in devolved systems. Audit essentially serves accountability purposes, 
but also provides information from which specific types of systemic performance 
indicators can be construe ted. Review, as indicated above, is important throughout 
all stages of the process of delivering high quality education to students. In fact, 
review — through continuous work place monitoring by every employee — is a basic 
component of the management of quality outcomes. 

There is a false belief held by some supporters of devolved systems that such 
systems can be held accountable on the basis of outcomes and outputs alone, thus 
substantially removing the need for the review function. This assumes that clients 
will exercise judgement about the quality of the service provided and adjust their 
demand for the service in light of this. While this perspective is built on sound 
consumer demand theory it is somewhat irrelevant to die production of a quality 
service. 

Three arguments can be put forward to substantiate the need for audit and 
review functions in a devolved state education system. First, accountability to 
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parliament requires not only that useful service is provided to clients, but that the 
service is efficient and provides value for money — state education is essentially an 
oligopoly in terms of markets, therefore, changes in client demand do not 
accurately reflect efficiency. The essentially 'free* nature of state education to the 
clieni means that any assessment based simply on consumer satisfaction provides a 
poor guide to quality, because a 'free' service provides no opportunities to assess 
the elasticity of demand for that service. 

Second, client-led accountability systems do not provide direct feedback as to 
why a service is providing or not providing quality service — that is, the client 
information does not crntain data on the characteristics of quality to which demand 
is responding. This is true even in unconstrained markets. In fact, producers of 
goods and services in competitive markets find it more — not less — important to 
undertake substantial product and service reviews. Consumer product testing is a 
very substantial industry, particularly in highly competitive markets. 

Third, the management — as opposed to the inspection — of quality requires 
continuous review throughout every aspect of development, production and 
delivery. A key feamrc of the Japanese approach to management is the assurance 
of the quality of the finished product by managing quality at every stage of 
production; and after delivery, in terms of continuing client satisfaction. It does not 
rely on inspection at the end of the production process to indicate that a product is 
defective — something, which incidentally, management has no interest in 
producing — and management does not rely on a decline in demand, as customers 
turn to other manufacturers models, to tell them that their product is of inferior 
quality. 

Aocountability or Development; TWo Masters or One? 

There is an interesting duality in the concepts of accountability and 
development applicable to education systems. Accountability refers to the proving 
of quality and development to the improving of quality. Clearly, systems based on 
input controls and those with a specific focus on outputs address the issue of 
accountability dircc Jy, although fi-om quite differert perspectives. However, they 
pay little attention to the contribution that accountability processes might make to 
the development needs of an organisation. The advantage gained by adopting a 
quality management perspective is that of binding the process of accountability and 
development into the one structure. Whilst it is important for operational reasons to 
maintain a distinction between accountability and development activities, it is also 
important for the overall effectiveness of an organisational unit that accountability 
and development are seen as complementary to each other. More importantly, 
accountability systems need to be established in a way which maximises their 
contribution to the development of organisational units. 
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Strategic Planning for School Developmeiit 

Over the past five years all Australian state education systems have developed 
strategic plans for the iniplementation of policies and programs. In the South 
Australian system such systemic plans are further augmented by development and 
operational plans for each of the directorates in the Department. During 1990 all 
schools established plans for their own development. These are known as school 
development plans (SAED, 1989, 1990a). School development plans are a 
statement of the key things which the school wants to change or improve 
(objectives); how these things are to be achieved (strategies); and what the impact 
will be (outcomes); to improve education for students. A school development plan 
is only one aspect of the school's total planning process. It describes the school's 
priority areas for developnn nt, but it docs not encompass the on-going maintenance 
activities of the school. The objectives as they are identified in the school 
development plan aiise from two sources: the programs and policies which the 
Education Department has mandated for implementation in schools, and the 
particular aspirations of schools and their communities. Action plans are developed 
by the school to address each of the objectives stated in their school development 
plan. The processes by which the school development plan identifies its objectives 
and implements them are crucial to the success of the plan. The objectives 
themselves and the implementation process must focus as directly as possible on 
increasing educational outcomes for students. 

The systemic strategic plan and directorate plans, along with school 
development plans, have a three-year time horizon and each is rolled-forward after 
their annual review of progress. 

Links to School Restructuring 

Education reforms in Australia, New Zealand, North America and United 
Kingdom in the last two or three years have led to an increased emphasis on site- 
based management and the devolution of funding to schools. These reforms have 
sought to give schools greater control over the way in wiiich they operate. This 
includes greater devolution of powers to schools over budgets and an associated 
increase in authority for site-based management. The most far reaching of these 
reforms have been in New Zealand and the United Kingdom. 

Experience in several countries has indicated that teachers have been somewhat 
over burdened by the trend which emerged in the 1980s of expecting them to 
re-invent the wheel of curriculum anew in every school and classroom. The 
present reforms have, therefore, also attempted to redress this situation by 
emphasising broader based collaborative and system- wide curriculum development 
initiatives. 



Feiformance Indicators in Education 



The literature on performance indicators in education makes the distinction 
between those which arc performance indicators, and those which arc education 
indicators. Tlliis perspective is particularly prominent in the North American 
literature. However, there is good reason to treat all indicators in education as 
performance indicators. The quality management approach described i ^. earlic 
sections jf this paper is based on the perspective that within any part of the 
educaticyn system an operational unit or an individual worker is simultaneously a 
client of the services provided in suppon to them from other parts of the system and 
a pnxiucer of services or products to other parts of the system. Hence, at any one 
point in the structure the operation of a unit or an individual can be viewed as a 
perf ormance in the context of the requiremenfs of their clients. 

Indicator systems in education have been proposed in order to address a range 
cf different issues. The main uses that have been suggested for them include: 

• assessing the impact of educational reforms 

• informing policy makers of the practices that are most effective for 
improving education 

• explaining causes of conditions and changes 

• informing decision making and management 

• stimulating and focussing effort 

• ensuring accountability 

• defining educational objectives 

• monitoring standards and trends 

• forecasting future changes. 

Each of the above uses of indicator systems in education is discussed in more 
detail in Cuttance (1989). Oakes (1986) has suggested that there are five types of 
information that indicators must provide at the operational level. These are: 

• performance information in relation to the achievement of goals and 
objectives 

• information on the features of tiie system that are most important in 
achieving particular goals and objectives 

• policy relevant information 

• problem orientated information 

• information on central features of tne system. 

Table 1 cross-classifies these five types of information against the nine 
purposes set out earlier for education indicator systems. The five types of 
information can be sub-divided into those which are derived fitt>m evaluations of the 
system and those which arc derived from routine monitoring of the system. The 
category of 'policy relevant information' is applicable to information from both 
evaluation and monitoring sources. Also, a certain amount of 'problem oriented 
information' will be made available through the diagnostic and formative 
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components of formal evaluation activities, but its main source will be from 
monitoring activities in the system. From table 1 it is clear that some of the 
puip^^s put forward for education indicator^ draw more heavily on monitoring 
activities and others draw more heavily on evaluation activities. Thus, an indicator 
system which encompasses all nine purposes would need to gather information 
from both formal and informal evaluations and from routine monitoring of the 
system. 

Table 1 Purposes of education indicator systems and the types of information they need 
to provide 
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The purposes of performance indicators described above are subsumed under 
the more general functions of quality assurance, development and accountability in 
education systems discussed earlier in this paper. 
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strategic Information to Assist the Development of Schools 

The review program conducted by the South Australian Education Review Unit 
aims to:' 

• provide direct feedback to individual schools and units on their education 
developnnent plans, and in the broader context of system wide reviews 

• provide timely information on the effectiveness of educational practice and 
the adequacy of the services provided 

School reviews contain four components: 

• A review of progress against the school development plan. Both the 
effectiveness of the strategics followed by the school in achieving the 
objectives set out in its plan, and the progress that it has made towards 
achieving those objectives are reviewed. 

• An audit of selected Education Department regulations and requirements. 
Regulations and requirements in die areas of general school management 
and organisation, curriculum management and policies, professional 
obligations of school staff to students, and school-community 
relationships are audited. 

• A school-initiated element This provides the school with an opportunity 
to request assistance of the review team in the review of a particular issue 
that is of interest and importance to the school. 

• An element of a systemic review. Systemic reviews include effective 
practice reviews, and program and policy reviews. Effective practice 
reviews are discussed further below. Program and policy reviews focus 
on the delivery and effectiveness of particular services and policies. 
Program reviews may evaluate a service directed towards the entire student 
population or, more frequently, to one or more groups within it. Program 
and policy reviews typically take the form of program evaluations which 
gather evidence across a number of schools and in othei parts of the 
system also responsible for delivering that program or policy to schools. 
They involve a omsideration of both the effectiveness of the program as it 
is delivered in schools and the support provided to schools by other 
development and service units with a responsibility for that program or 
policy. (SAED, 1990b,c, 1991) 

Efifective Practice Reviews 

The primary purpose of this type of review is to report on performance with 
respect to particular practices across tlic system as a whole. Effective practice 
reviews are conducted in four domains: teaching and learning, organisation and 
management, ethos and culture, and equal opportunities and social justice. Each 



> The Education Review Unit is not responsible fcM- student assessment The Curriculum 
Directorate is currently developing new procedures for assessing students—a series of six 
'attainment levels* from year 2 to year 10. The ERU does, however, incorpOTaie assessments of 
the quality of student work in schools in its reviews of the performance of schools. 
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domain addresses five to eight areas of school activity. Three to five aspects of 
performance have been developed to report on each of the areas of activity 
reviewed. In total this provides about 100 profiles of performance across the four 
domains of practice. In general, only one of the four domains of effective practice 
is reviewed in each school. 



Figure 1 The four domains of effective practice reviews 




The effective practice instruments were developed by icturs of school-based 
teachers, principals and superintendents. In drawing up the statements of effective 
practice in each of the aspects to be reviewed these teams drew upon the research 
literaiure in the relevant area, the Department's guidelines and policy statements 
relevant to that aspect, and their own experience as educators in schools. The 
instruments were then trialled in various reviews and revised before being 
incorporated into the review process in schools. Annual revision of the instmments 
is envisaged. Appendix 1 provides a listing of the areas and aspects of practice and 
functioning in each of the four domains and Appendix 2 provides an example from 
the effective practice instruments for each of the four domains. 

Issues raised in developing the instruments reflect some of the current debates 
about different forms of student assessment. In particular, the approach adopted in 
writing the statements of effective practice parallel many of the characteristics 
associated with authentic pupil assessment (Shepherd, 1991). 

• Gathering information with the instruments is judge or observer intensive. 

• Samphng is used to reduce overall costs — the system rather than the school 
is the unit of analysis. 
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• They have a less distorting effect on school practice than check lists of 
practice. 

• Both processes and outcomes are observed as part of the review. 

• The trade off between depth and breadth of practice observed had to be 
taken into account in their development. 

• The instruments are not meant to provide prescriptive or comprehensive 
statements of the characteristics that might be associated with all effective 
practices, rather they describe some of the key characteristics that one 
would expect to observe in effective teaching and learning situations. 

The gathering of the information in effective practice reviews is context 
sensitive — ^in particular, consideration is given as to whether ot not it is appropriate 
to expect a particular nrgctice to occur in the situation being observed. 

The ERU monitors and revises its descriptions of effective practice in light of 
feedback from practitioners in schools and from an evaluation of the relationship 
between vaiious practices and student learning. The latter builds on departmental 
approaches to student assessment to evaluate the relative importance of different 
practices for the educational development and progress of students. 

The information obtained from effective practice reviews is used to diagnose 
weaknesses and monitor trends in the education system as a whole. Further, it 
provides summative information on the standard of practice that exists in the 
system. This information provides important input into policy formulation, and 
operational decision making and resource allocation. It also contributes to the 
annual revision of the Department's systemic Three year Plan. 

The information obtained in effective practice reviews is also linked to 
departmental programs and policies. A two dimensional matrix with programs and 
policies on one axis and effective practice indicators on the other establishes the 
linkage between individual aspects of effective praciice and particular programs and 
policies. Thus, effective practice reviews provide an on-going basis for monitoring 
the effectiveness of individual programs and policies. However, the range of 
effective practices reviewed is broader than the set of systemic programs and 
policies. That is, various aspects of practice that arc not specifically addressed by 
programs or policies are also monitored. 

The effective practice review instruments are also being prepared for use in 
other contexts in tlie system. In particular, some schools have expressed an interest 
in using them as an internal screening device, and other schools have expressed an 
interest in using them in staff trai.iing and development sessions. 
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Reviewing PeifonniLnce of School Practice and Functioning 

Data CoUection in Schools 

In each school review only one domain of effective practice is reviewed. 
Information is collected from staff and a cross-section of parents and students, 
mainly through interviews and discussions. In addition, relevant documents may 
be examined However, direct observation also pays an important part in gathering 
information. Observation by the review team of the general environment of the 
school, student recreation and study areas, classrooms, staff-rooms, etc, is 
undertaken. Depending on the domain of practice being reviewed in a particular 
school, discussions may also be held with specific individuals, such as those 
responsible for the financial management of the school, those with key 
responsibilities for cuuiculum development and implementation, etc. The types of 
documents that may be perused as part of the information gathering CACrcise 
include: school newsletters, parent and staff information booklets, teacher work 
plans, student records and reports, minutes of school meetings, etc. 

The domains of management and organisation, ethos and culture, and equal 
opportunities and social justice use the school as the unit for reporting. Reporting 
on the domain of teaching and learning is based on the classroom as the unit of 
observation. This latter domain involves considerably more classroom observation 
than the other three domains. Typically, in reviewing teaching and learning review 
team staff observe in classrooms for between half an hour and fifty minutes on each 
occasion. They involve themselves in the classroom to the extent necessary to 
observe the practices being reviewed. This may include direct interaction with 
children in the classroom, with groups of children, with the teacher, etc. 
Normally, some discussion take place with the individual teacher before liic 
observation period and again a'ter the observation period. The purpose of the 
observation is to gather information about noraial practice in schools. No special 
preparation is required by teachers. The observation is not linked to any process of 
teacher appraisal. Hence, no feedback is given about the general effectiveness of 
the teacher, although the review team member may discuss some aspects of practice 
that were observed during the session with the teacher, if the latter so desires. 

Since the effective practice reviews have the specific purpose of improving the 
quality of practice and functioning in schools, the recording of information is based 
on categorical statements that relate to the extent of the implementation of the 
practice as an effective practice in that situation. Three decisions have to be made 
by the observer about each aspect reviewed. The first is whether or not there is 
sufficient data on which to make an assessment of the effectiveness of the practice 
associated with that aspect. The second decision is whether the practice in that 
aspect is regarded as effective (E) or not effective (N) as described by the 
description in the effective practice indicator statement. The third decision requires 
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a distinction to be made between levels of effectiveness (or ineffectiveness). The 
reviewer assesses whether the practice observed is regarded as effective at an 
outstanding level (EO) or at a satisfactory level (ES). Similarly practice which is 
regarded as ineffective is classified according to whether or nor this has been 
recognised (by the teacher or the school) as an aspect of functioning which requires 
development and for which some planning for improvement is in hand (NCP), or 
whether that aspect of functioning has not yet been acknowledged as requiring 
improvement (NN).2 

An annual repoit based on the observations of school reviews in each of the 
domains is published. These reports do not identify individual schools or teachers. 
They highlight aspects of practice that have been found to be particularly effective 
and make recommendations where performance is considered to be weak generally 
in schools, or where the practice appears to be weaker in schools serving particular 
sections of the student population. These recommendations may lead to more 
indepth reviews of particular policies or programs which are responsible for 
supporting those areas of practice found to be weakest. Such further reviews may 
evaluate a particular program or policy, or a group of them, or they may focus on 
particular aspects of functioning and practices in schools. The analysis assists the 
effective utilisation of systemic resources for improvement in key areas and thus 
supports sch :k31s in their function of providing high quality education for students. 



Findings for the South Australian Education System 
Context 

The South Australian state education system is divided into five operational 
areas and there are three central directorates (Resources, Personnel, Curriculum) 
plus the Education Review Unit. There are approxiuiately 738 schools in the 
system: about 100 secondary schools serving years 8-12, 54 country (ai-ea) and 
urban Reception - year 12 schools, 424 primary schools serving Receptior. - year 
7, 77 junior primary schools serving Reception - year 2, 67 schools for special 
students, and 16 schools for Aboriginal students. 

There are approximately 185,000 students, 16600 teaching staff, 4700 other 
staff, 300 out of school support staff, and 900 staff in administration in the system. 



^ Note that the categorisation of effectiveness reponed for the data reported in later sections of this 
paper differs from that described here. The recording schema has since been revised in line with the 
above. 

The assessment of performance for the teaching and learning domain is undertaken by observers 
independently for each classroom they visit. The assessments for the other three ''omains arc 
arrived at through discussion and consensus among the members of the review team. 
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Figure 2 Development of schools by level of schoolingt 
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^ Because of the small number of schools reviewed at some levels of schooling, care must be exercised in 
interpreting the differences in these assessments. The estimates should be read as indicating the range of 
variation between different types of schools. These arc aggregate group statistics, therefore, they do not 
indicate the assessment pertaining to specific schools in a group. 

The Education Review Unit review program commenced in term 2 1990. One 
hundred and twenty five schools were reviewed during terms 2-4. In the process 
of gathering information in these reviews, approximately 1500 teaching staff, 300 
ancillary staff, 2000 parents, and 4000 students were involved in discussions and 
individual interviews. In addition, over 500 teaching lessons were observed. The 
overall assessment from the reviews indicated that 42 percent of schools reviewed 
were well advanced in terms of their development planning and were able to 
effectively manage and sustain their own development Thirty percent were found 
to be able to establish the necessary structures and processes for sustainable 
development with the normal assistance and support provided by arcu and system 
programs. Seventeen percent were found to be in need of early and continuing 
suppon to establish the necessary structures and processes for sustainable 
development. Twelve percent of schools were found to be in need of substantial 
support over a prolonged period in order to establish the structures and processes 
for sustainable development. Significant leadership and organisational 
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development and change is likely to be necessary to establish the basis for effective 
development in this latter group of schools. 

Domain: Teaching and Learning 

The following analysis is based on infonnation collected by means of 
observation in 80 randomly selected classrooms, and through discussions with the 
teachers in whose classrooms the observations were undertaken. I-ractices were 
found to reflect the indicators of effectiveness either \ery closely or fairly closely, 
in 60 percent of the classrooms observed for 20 of the 21 aspects. Appendix 3, 
provides a graphical summary of the findings for the aspects of practice reviewed in 
this domain. 

Highly Effective Practices 

Practices were found to be effective or highly effective in 80 percent or more of 
the classrooms in five aspects. Each of these is discussed below. 

Classroom Interpersonal Interactions 

There was evidence of sound interpersonal relationships in 84 percent of the 
classrooms amongst students and between the teacher and students. Interactions 
were characterised by respect, interest, sensitivity, consideration, trust and 
humour. Effort and success were acknowledged and instances of disruption or 
conflict were dealt with constructively. 

Home/School Information Flow 

Findings in this aspect indicate tliat formal and informal structures for parents 
and teachers to share information were used effectiv.;ly in 84 percent of the 
classrooms. In these classrooms, teachers were successfully communicating 
information about important aspects of schooling to parents and were managing 
sensitive information conveyed by parents in appropriate ways. 

Teacher/Learner Relationships 

Relationships were seen to be highly effective in 47 percent of classrooms and 
effective in a further 38 percent. There was evidence of teacher's concern for 
students and of encouragement for them to do their best. These classrooms 
demonstrated a positive approach to learning and teaching; expectations were 
appropriate and clear. Teachers conveyed their interest in students as individuals. 

Student Attitudes and Enthusiasm 

Students set to work quickly, concentrated on the task at hand and made an 
effort to produce work they could be proud of. They demonstrated, enthusiastic 
and confident approaches to learning activities to a high or very high degree in 81 
percent of classrooms. 
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Monitoring and Reporting Progress 

Teacher practices in this aspect were found tc be highly effective in 23 percent 
of classrooms and effective in a further 58 percent. In these classrooms teachers 
monitored student behaviour and participation and took appropriate steps to ensure 
that an effective working environment was maintained. These teachers kept useful 
records of student progress and had developed appropriate ways of reporting to 
students and parents about this. 

Practk^es Requiring Devekipment 

In one of the 21 aspects more than 40 percent of classrooms indicate practices 
which were seen to be ineffective. In several other aspects, however, review 
findings suggest that there are matters which warrant attention. 

Curriculum Monitoring and Review 

Findings indicated that the curriculum monitoring and review practices in 5 1 
percent of classrooms do not meet the described criteria of effectiveness. There 
was no established process for checking that aims and objectives are appropriate to 
students or for monitoring the effects of curriculum content, resources and use of 
time on student learning in these classrooms. The absence of a systematic process 
of monitoring and review limits the extent to which teaching can be modified to take 
account of students' individual differences. It also inhibits the extent to which 
parent and student perceptions of the effectiveness of learning can be taken into 
account when learning programs are being designed. 

These findings are consistent with information that was provided by principals 
during the ^evie^^ of the Curriculum Authority and Responsibility policy. 

Other Aspects for Attention 

There were four other aspects of practice which came to attention. These were: 

• quality of teaching skills 

• organisation for learning 

• appropriate curriculum resources 
use of learning resources. 

In these aspects, practices in more than 60 percent of classrooms were rated as 
effective, but in 6-10 percent of classrooms practices were rated at the lowest level 
of effectiveness. 

The indicators of effectiveness in the first two of these aspects refer to the 
range of teaching approaches and groupings that are used 

• to provide appropriate sequencing and continuity of learning for students 

• to make opp<xtunities for teachers to nxxiel and students to practise a range 
of learning strategies 

• to facilitate high quality teacher interaction with students, so that all 
students are involved in making decisions about their learning 
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• to give practical expression to concepts such as the valuing of difference, 
exploration of ideas, excitement in learning and the pursuit of excellence. 

Indicators of effectiveness in the latter two aspects identified above refer to the 
range of resources, both human and material, in the school and the wider 
community, which are used to ensure that learning experiences adequately support 
differences in student abilities, learning styles, interests and cultural backgrounds. 

Summary 

The review found that for most aspects of classroom functicning, practices 
were effective in the vast majority of classrooms, and in a considerable proportion, 
were very effective. The fmdings indicate that procedures for the systematic 
monitoring and review of the cuiriculum need further development in at least 50 
percent of classrooms. There is some evidence to suggest that practices which 
effectively support students* development of learning strategies arc less evident 
than those which support their social development. There is also an indication in 
the findings that more attention should be given in some classrooms to the use of a 
wider range of resources and teaching approaches to acknowledge and support 
student differences. 

Management and Organisation 

This analysis is based on information collected in a random sample of 17 
schools. More than 50 percent of schools were found to match the descriptions of 
this effective practice instrument either very closely or fairly closely for 17 of the 30 
aspects reviewed. 

Highly Effective Practices 

Grounds, Facilities and Equipment 

No school in the sample was assessed as being weak in this aspect of effective 
practice. Eighty percent of schools were assessed to match the indicator very 
closely OT fairly closely. This is an aspect of schools where most have effective 
council and staff sub-committees that regularly report on grounds development and 
maintenance and on financial activity within the school. 

Inclusive Attitudes and Practices 

The indicator statement for this aspect is of a general character and many of the 
individual factors within the aspect are explored in more detail in the domain of 
equal opponunity and social justice. In approximately 75 percent of schools the 
differences between individual students is recognised and valued. Enrolment is 
open to students regardless of cultural or linguistic background, physical or 
intellectual impairment (save those directed by the Director General of Education to 
attend a special purpose school/unit) and processes to access appropriate support 
arc operative. 
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Staff Leadership Positions 

In approximately 70 percent of schools the practice matched the indicator 
statement for effective practice either very closely or fairly closely. Leadership 
positions have been allocated to meet the objectives of the school development plan. 
However, the success of the training and development associated with these 
positions to achieve planned objectives was commented on as one of the Ic'^s 
effective practices in these schools. 

Management of Resources Beyond the School Environment 

In approximately 60 percent of schools the education program is planned 
sufficiently in advance so that members of the school community are aware of 
approaching events, costs and the organisational requirements. In these schools the 
principal is aware of Education Department processes, has an effectively 
functioning network of appropriate resource contacts, and uses this network to the 
school's advantage. 

Working Environment 

Approximately 64 percent of schools have staff who show high esteem and 
who work collaboratively to achieve high standards. The staff take responsibility 
for the success or failure of their projects and effective co-operation is clearly 
evident. 

Budget Management 

Approximately 60 percent of schools have practices that fairly closely match 
the effective practice. Structures and processes relating to financial management are 
in accord with the document Financial Management in Schools. In these schools a 
degree of financial flexibility is achieved and regular reviews are built into the 
budget cycle. 

Less Effective Practices 

Internal School Review 

The lowest degree of effectiveness was evident for this aspect of practice. 
School development planning is currently in its implementation phase, and may 
schools have little experience in conducting internal school reviews. By the end ot 
1991 it is expected that all schools will have conducted an internal review as part of 
the roll-forward of their school develc^ment plan. This experience will provide the 
basis for on-going learning and subsequent improvement in practice. 

Curriculum and Program Evaluation 

No school was assessed as having matched the description in the indicator 
statement very closely ot fairly closely. This assessment is consistent with the 
information provided by the review of the Curriculum Authority and Responsibility 
policy. The inclusion of the statement "tiiat staff are continuously involved in 
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reviewing ckssroom practices and programs'* indicates the high level of total school 
involvement that is required in this aspect of practice. A high degree of trust is 
necessary before staff feel comfortable with the involvement of peers in the 
assessment of their classroom practice. 

Staff Supervision and Support 

No school matched the description of practice in the indicator statement very 
closely in this aspect The key features of this aspect are a school supervision and 
support policy, with encouragement to cry different options, and the linking of staff 
supervision to the achievement of school development plan objectives. The 
reviews noted that school staff are waiting for the develc^ment of the Department's 
policy on perfonnance appraisal before developing school-based practices. The 
Department's policy should be developed at an early opportunity. The current 
standard of practice indicates that major training and development will be required, 
as the base level of practice is weak and provides litUe to build on. 

Training and Development 

Sixty five percent of the observed practice matched the indicator statement in 
some respects. This suggests that although appointments are being made to the 
new leadership positions, they are not yet resulting in effective change. The 
succer^sful implementation of change requires multiple strands within the schools 
training and development program. An essential component is leadership 
knowledge of individual teacher performance to facilitate counselling to increase 
skills for career advancement prospects. The assessment above of weak practice in 
the aspect of staff supervision and support strengthens this diagnosis of the 
problems in this aspect of practice. 

School Management Planning 

The weak assessments for this aspect arc congruent with the large number of 
recommendations relating to decision making and management jirocesses in school 
review reports. There may have been a lessening of emphasis on on-going 
management activities of a consequence of the increased emphasis on school 
development planning over the past year. The more effective principals in schools 
continue to manage curriculum maintenance but others are not consistent in their 
articulation of these management necessities. 

Curriculum Review 

No school was assessed as matching the indicator statement very closely which 
is consistent with the findings from the Curriculum Authority and Responsibility 
review. The indicator statement specifies that tiie cumculum must be consistent 
with the school's vision statement. The development of vision statements receives 
a high rating elsewhere in the review of this domain. However, it appears that few 
schools assess the congruence between the vision statement and current practice 
when determining their priorities for school development 
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Suminaiy 

The performance of management in the use of resources beyond the direct 
school environment and of buildings and grounds is highly effective in most 
schools. Aspects of management relating to staffing and students are performed 
effectively in more than half of all schools. Some specific aspects of general 
management — ^particularly those relating to decision making, staff training and 
development and the supervision and support of staff — ^require attention in more 
than half of all schools. Aspects of management relating to school planning — 
particulariy, context setting for planning, school management planning and internal 
review processes for school development planning — also require early attention in 
more than half of all schools. There is a weakness across the system in relation to 
aspects of curriculum management — curriculum review and relevance, and 
program evaluation. 



Discussion 

The system of school and system review that has been established in South 
Australia aims to directly address and integrate both development and accountability 
issues. The review structures and processes have been constructed in the light of 
the requirements of quality management, as exposed to quality control, systems. 

Systemic performance indicators of effective practice have been developed as 
one component of the broader review structure. These provide the basis for 
monitoring the effectiveness of practice and functioning in schools and other 
organisational units in the system. The effective practice indicators are designed to 
provide a basis for monitoring the system's programs and policies as they are 
delivered to students. 

Information from the effective practice performance indicators is available for 
only a small number of schools thus far. The information available, however, 
suggests certain weaknesses and strengths in the system. 

The assessment of the information from the domains of teaching and learning 
and management and organisation indicates that the effectiveness of practice and 
functioning in the latter domain is weaker than in the former, the information 
available from the performance indicators for both domains indicates that strengths 
are found in the non-curriculum aspects of schooling and that future attention in the 
development of schools needs to be directed towards curriculum issues: 
particularly, planning, maintenance, development and review of curriculum and 
programs within schools; the organisation and strategies for the effective use within 
schools of resources to deliver tiie curriculum, including staff training and 
development; the management and review of development in schools. 
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Once the parallel analyses of the effectiveness of performance in the other two 
domains of ethos and culture and equal opportunity and social justics have been 
COTipleted the infonnation will be utilised for the following purposes: 

• to assess which programs and policies may be performing at less dian 
satisfactc^ levels 

• to indicate areas of strength in the delivery of high quality education to 
students 

• to indicate the aspeas of practice aiKl functioning in schools that require 
priwity attention, and therefore 

- appropriate targets for school improvement to be addressed through 
school develq)ment plans and the strategic plan for development across 
the system 

- the areas of priority for staff training and development 

• the assessment of progress towards the attainment of individual objectives 
on the systemi" Three Year Plan. 
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